8th meeting of the medical assessment consortium UCAN: “Collaborative Perspectives for Competency-based and Quality-assured Medical Assessment”

Text On February 9 th and February 10 th 2015, the German-speaking members of the " Umbrella Consortium for Assessment Networks (UCAN) " met at the University Medical Center Göttingen for the 8 th annual partners meeting. At UCAN, more than 60 institutions from 7 countries work together to create and share new exam contents and formats as well as to support each other in the quality assurance and further development of this content. This year's meeting focused on " Collaborative Perspectives for Competency-based and Quality-assured Medical Assessment ". More than 80 participants from partner institutions as well as external guests shared their experiences about competency-based assessment, collaborative quality assurance and assessment technology in 12 workshops and numerous plenary sessions. In order to effectively deal with the current challenges, (such as the implementation of the National Competency-based Learning target Catalog in Medicine (NKLM) and the recommendations of the German Council for Science and Humanities) the consortium's joint assessment research work shall be intensified with the establishment of an " advisory board for collaborative research ". Shaping tomorrow's assessment culture through cooperation " Non scholae, sed vitae discimus " – we learn for life, not for school. With this guiding principle Prof. Dr. Gerhard Burckhardt, Study Dean of the Medical Faculty Göttingen, opened the 8 th UCAN partners meeting and reminded the audience that assessment isn't an end in itself. Highlighting the importance of high quality competency-based exams for a good medical education culture, Burckhardt outlined an agenda as to how students could be enabled to " learn for life " instead of " learning for school " (see figure 1).


Text
On February 9 th and February 10 th 2015, the Germanspeaking members of the "Umbrella Consortium for Assessment Networks (UCAN)" met at the University Medical Center Göttingen for the 8 th annual partners meeting. At UCAN, more than 60 institutions from 7 countries work together to create and share new exam contents and formats as well as to support each other in the quality assurance and further development of this content. This year's meeting focused on "Collaborative Perspectives for Competency-based and Quality-assured Medical Assessment". More than 80 participants from partner institutions as well as external guests shared their experiences about competency-based assessment, collaborative quality assurance and assessment technology in 12 workshops and numerous plenary sessions. In order to effectively deal with the current challenges, (such as the implementation of the National Competency-based Learning target Catalog in Medicine (NKLM) and the recommendations of the German Council for Science and Humanities) the consortium's joint assessment research work shall be intensified with the establishment of an "advisory board for collaborative research".
Shaping tomorrow's assessment culture through cooperation "Non scholae, sed vitae discimus" -we learn for life, not for school. With this guiding principle Prof. Dr. Gerhard Burckhardt, Study Dean of the Medical Faculty Göttingen, opened the 8 th UCAN partners meeting and reminded the audience that assessment isn't an end in itself. Highlighting the importance of high quality competency-based exams for a good medical education culture, Burckhardt outlined an agenda as to how students could be enabled to "learn for life" instead of "learning for school" (see figure 1). Prof. Dr. Burckhardt's appeal has the finger on the pulse of medical assessment: instead of only testing knowledge, the evaluation of clinical competencies becomes increasingly important. Especially the implementation of the NKLM and the recommendations of the German Council for Science and Humanities are major challenges for the (German) faculties. Consequently, the main question for the 12 workshops was: How can the turn towards competency-based assessment be implemented efficiently without neglecting the quality assurance?
Intensifying collaborative research within the consortium Stephanie Seidemann (Heidelberg), a competency-based assessment program for palliative medicine was developed, which found the consensus of all attendees. It also became clear that the implementation of such a program can only be achieved together as a network of partners. This would be the best way to tackle most of the currently existing problems like the lack of sufficient working hours dedicated to palliative education or the lack of formats to teach and assess a professional attitude in palliative medicine. In a follow-up session on day two, the group developed first formats for assessing communicative skills in palliative medicine. In order to further intensify the joint research in the consortium, an "advisory board for collaborative research" was established at this year's meeting. The board consists of the assessment coordinators and the designated persons with the qualification of a Master of Medical Education (MME) from the UCAN partner faculties (see figure  2). Further strengthen collaborative quality assurance

During the workshop "Quality Assurance with the Help of Faculty Reports and the Standardization of Evaluation
Methods" under the guidance of Dr. Andreas Möltner (UCAN) and Dr. Irmgard Streitlein-Böhme (Freiburg), participants discussed how the shared exam platform Item-ManagementSystem (IMS) can be utilized for collaborative quality assurance. Besides discussing the existing features, the participants also made suggestions for an easier graphical overview of the test statistics as well as a longitudinal comparison of cohorts with the help of a graphical display.
In order to further improve the quality of the +200.000 items stored in the database, Dr. Tobias Raupach (Göttingen) conducted a training session titled "Good MC Questions with the Help of the IMS Review and the Blueprinting System", in which examiners and item authors were trained how to write good MC questions and how the reliability and validity of exams could be enhanced with the IMS quality assurance functions. One slot was dedicated to an open exchange for the German Chambers of Physicians (Ärztekammern) organized in UCAN. One of the main topics was how the participating chambers could use the integrated IMS review system for their quality assurance requirements.
In the workshop "Competency-based Student Progress Test" under the guidance of Stefan Wagener (Heidelberg) and Felicitas Eckrich (UCAN), students had the chance to give feedback on the recently conducted progress test and how the student feedback after the test could be improved. The Student Progress Test is a format in which specially trained students create and review the exam content themselves. To generate test questions, "subject groups" (based on the medical licensure act, ÄAppO) and "competency areas" (based on the NKLM) were combined to create a two-dimensional blueprint.

Large interest in tablet-based exams
In various technical workshops, the UCAN Partners could learn more about the latest technical developments and new features of the UCAN Tools. During one session, participants learned how to efficiently use the new features of the Examinator for running their post-exam evaluation and test statistics as well as how to use the integrated scanner tool "EDGAR". In another session, the new features of the IMS were presented, especially how items can be authored with the completely redesigned item editor. Dr. Manfred Herrmann from the University Medical Center Göttingen also gave a presentation on how they plan, prepare and conduct desktop computerbased electronic exams with the UCAN Tool "Campus". During these sessions, the developers also got important feedback from the users as to how the system could be further improved.
One of the most-frequented events of this year's conference was the workshop "Conducting Exams on Tablet Computers" under the guidance of Jörn Heid (UCAN). Apart from the new features of the well-established tOSCE app for tablet-based OSCEs, the UCAN developer team also presented the new app "tEXAM": with this app, Partners can deliver MCQ exams created with the IMS on tablet computers. Various partners gave a presentation on their first experiences with this new app. In addition to the 15 partners that already use tOSCE, many more partners have announced that they want to switch from paper-based checklists to tOSCE for their practical exams (see figure 3).

new partners join UCAN
In the plenary, six new partners that have recently joined UCAN gave a short presentation about their assessment programs:

th anniversary meeting next year in Heidelberg
The partners meeting concluded with a final plenary session, in which the main results of the conference were summarized. The final documentation of the conference can be obtained from the UCAN office on request.
Next year's conference will be the 10 th anniversary of UCAN and is going to be held in Heidelberg. More information on the meeting as well as on the medical assess-ment consortium UCAN can be found at www.uc an-assess.org.